Watermarking of digital images using watermark-indicators

ABSTRACT

Methods and systems for watermarking of digital images are presented. In one aspect, a method of embedding information in a digital image includes transforming the digital image to a set of coefficient blocks having coefficients in a frequency domain, embedding a watermark-indicator in one or more of the coefficient blocks, and embedding a watermark in one or more watermark blocks that have a predetermined number of coefficient blocks. The embedded watermark is substantially invisible in the output watermarked digital image. Another aspect is a method of extracting a watermark from a digital image that includes transforming a digital image to a coefficient matrix, determining if the digital image is watermarked based on a predetermined watermark-indicator, and retrieving a watermark from a projected watermark block of a projected watermark block distribution.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional applicationSer. No. 12/354,534, filed on Jan. 15, 2009, now allowed, which ishereby incorporated by reference in its entirety.

BACKGROUND

1. Field

This invention relates to watermarking of digital images.

2. Background Art

Digital imaging allows images to be represented with digital data. Withthe growth in availability and access to the World Wide Web (WWW) andthe widespread availability of digital imaging technology, the number ofdigital images and the frequency of their use has grown. Manyapplications provide accessibility to particular digital images or imagecollections to a large number of different users for purposes such asviewing, editing, or sharing. For instance, digital images of persons,animals, objects, and scenery, are often posted on the WWW by owners ofthose images with the intent of sharing the images with a select groupof people or the public at large.

However, the accessibility of these digital images on the WWW makes iteasier for others to copy and use them for unauthorized purposes. Forexample, a member of Google's Orkut social networking community may postan image to identify himself as the owner of material posted on hisblog. Another person may copy this image and post it elsewherepurporting to be his own. Many examples exist, such as above, wheredigital images can be copied and used by persons other than the rightfulowners.

One technique used to reduce unauthorized copying of images, is thetechnique of watermarking. The watermarking of a digital image includesthe embedding of a digital watermark (referred to simply as a“watermark”) in that image. The watermark can be a digital signature, auser identifier, a logo, or similar value that can sufficiently identifythe image as to its origin or ownership. It is highly desirable that theembedded watermark does not alter the appearance of the original image,i.e., the watermark should be substantially invisible to the human eye.The watermark perceivably alters the appearance of the original image inmany of the conventional watermarking methods.

Digital images are generally posted on the WWW in a compressed form,such as, the Joint Photographic Experts Group (JPEG or JPG) compressionformat or other type of image compression format. Because imagecompression methods alter the values of selected image pixels, it ispossible that the embedded watermark is affected in such a way as toalter the appearance of the image and/or to make the watermarkunrecoverable. Therefore, it is also desired that a watermark be robustto survive high levels of compression, such as JPEG 60 or JPEG 80compression.

Often, a person copying a digital image also engages in altering theimage, for example, by cropping the original image. In order to beeffective, an embedded watermark should also be detectable and/orretrievable from a copied image after such cropping has taken place.

Many approaches to digital watermarking have been proposed. For example,one approach involves the generation of a discrete courier transform(DCT) coefficient matrix (DCT matrix) for the image, dividing thecoefficient matrix into one or more watermark blocks, and embedding adigital signature in each of the watermark blocks. Wen-Nung Lie,Guo-Shiang Lin, Chih-Liang Wu, and Ta-Chun Wang, “Robust ImageWatermarking On the DCT Domain,” IEEE International Symposium onCircuits and Systems, Geneva, (2000), describes such an approach. Themethod described by Lie et al. embeds a watermark repetitively indifferent sections of an image in the corresponding DCT matrix. However,this method is vulnerable to cropping and is computationally expensive.For example, cropping a watermarked image having a watermark embeddedusing the Lin et. al. method, can prevent the cropped image from beingrecognized as a watermarked image. Also, the method described by Lie et.al., incurs a relatively high cost of computation by requiringidentification of a substantial part of the originally embeddedwatermark in order to determine that the image was watermarked.

Therefore, improved methods and systems for watermarking of digitalimages are needed.

BRIEF SUMMARY

Methods and systems for watermarking of digital images are presented.One embodiment is a method of embedding information in a digital imageto create a watermarked digital image. The method includes transformingthe digital image to multiple coefficient blocks of coefficients in afrequency domain, embedding a watermark-indicator in each of thecoefficient blocks, embedding a watermark in each of the watermarkblocks to generate a watermarked digital image, and outputting thewatermarked digital image.

Another embodiment is a method of extracting a watermark from a digitalimage. The method includes transforming the digital image to acoefficient matrix having coefficients in a frequency domain,determining if the digital image is watermarked based on a predeterminedwatermark-indicator coefficient set in each coefficient block in aprojected coefficient block distribution, and retrieving a watermarkfrom a projected watermark block of a projected watermark blockdistribution. The projected watermark block distribution is based on theprojected coefficient block distribution.

Yet another embodiment is a system for watermarking a digital image. Thesystem includes a processor, a memory, an image transform module, and awatermark embedding module. The image transform module is configured totransform the digital image to a plurality of coefficient blocks ofcoefficients in a frequency domain. The watermark embedding module isconfigured to embed a watermark in each of a plurality of watermarkblocks to generate a watermarked digital image. The watermark-indicatorembedding module is configured to embed a watermark-indicator in each ofa plurality of said coefficient blocks.

A further embodiment is a system for extracting a watermark from adigital image, having a processor, a memory, an image transform module,a watermark-indicator detection module, and a watermark extractormodule. The image transform module is configured to transform thedigital image to a plurality of coefficient blocks of coefficients in afrequency domain. The watermark-indicator detection module is configuredto determine if the digital image is watermarked based on apredetermined watermark-indicator coefficient set in each coefficientblock in a projected coefficient block distribution. The watermarkextractor module is further configured to retrieve a watermark from aprojected watermark block of a projected watermark block distribution.The projected watermark block distribution is based on the projectedcoefficient block distribution.

Further features and advantages of the present invention, as well as thestructure and operation of various embodiments thereof, are described indetail below with reference to the accompanying drawings. It is notedthat the invention is not limited to the specific embodiments describedherein. Such embodiments are presented herein for illustrative purposesonly. Additional embodiments will be apparent to persons skilled in therelevant art(s) based on the teachings contained herein.

BRIEF DESCRIPTION OF THE DRAWINGS/FIGURES

Reference will be made to the embodiments of the invention, examples ofwhich may be illustrated in the accompanying figures. These figures areintended to be illustrative, not limiting. Although the invention isgenerally described in the context of these embodiments, it should beunderstood that it is not intended to limit the scope of the inventionto these particular embodiments.

FIG. 1 is a flowchart of a method of embedding watermarks in digitalimages, according to an embodiment of the present invention.

FIG. 2 shows a coefficient array representation of an image, accordingto an embodiment of the present invention.

FIG. 3 shows example locations in an array representation of an imagewhere watermark information can be embedded, according to an embodimentof the present invention.

FIG. 4 shows a flowchart of a method of embedding watermarks in digitalimages, according to an embodiment of the present invention.

FIG. 5 shows a flowchart of a method to determine if a digital imagecontains a watermark, according to an embodiment of the presentinvention.

FIG. 6 illustrates a process for determining if an image is watermarked,according to an embodiment of the present invention.

FIG. 7 illustrates a process for extracting an embedded watermark froman image, according to an embodiment of the present invention.

FIG. 8 illustrates aspects of processing in for extracting an embeddedwatermark from an image, according to an embodiment of the presentinvention.

FIG. 9 is a system for watermarking images, according to an embodimentof the present invention.

FIG. 10 shows an image before and after being watermarked, according toan embodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

While the present invention is described herein with reference toillustrative embodiments for particular applications, it should beunderstood that the invention is not limited thereto. Those skilled inthe art with access to the teachings herein will recognize additionalmodifications, applications, and embodiments within the scope thereofand additional fields in which the invention would be of significantutility.

A watermark in a digital image can be used to identify the image andalso any digital copies of that image. A watermark can include anyidentifying criteria as to the origin and/or ownership of that image.For example, digital signatures, user identifiers, logos, codes, etc.,are some types of watermarks that are used in digital images.

Watermarks can be used in digital images in a variety of applications tohelp identify the origin or ownership of a digital image and to helpreduce unauthorized use of digital images. One example environment inwhich embodiments of the present invention can be used includes reducingthe unauthorized copying and posting of images on the WWW. For example,embodiments of the present invention may enable the embedding of awatermark comprising the user identifier of the original poster inphotographs posted on PICASA or on ORKUT, both web-based servicesoffered by Google of Mountain View, Calif. Whenever a watermarked imageis copied and posted elsewhere, embodiments of the present invention canallow any concerned entity to determine if that image was watermarkedand if so retrieve the embedded watermark. The knowledge that a postedimage is already watermarked permits a determination as to itsauthenticity. The retrieval of the actual embedded watermark permits theimage to be tracked to its source.

In embodiments of the present invention, methods and systems forwatermarking of digital images are presented. Example embodimentsinclude a method of embedding information in a digital image to create awatermarked digital image, a method of extracting a watermark from adigital image, and a system for watermarking a digital image. One methodof creating a watermarked digital image includes transforming a digitalimage to multiple coefficient blocks of coefficients in a frequencydomain, embedding a watermark-indicator in each of the coefficientblocks, embedding a watermark in each of the watermark blocks togenerate a watermarked digital image, and outputting a watermarkeddigital image. A method of extracting a watermark includes transformingthe digital image to a coefficient matrix having coefficients in afrequency domain, determining if the digital image is watermarked basedon a predetermined watermark-indicator coefficient set in eachcoefficient block in a projected coefficient block distribution, andretrieving a watermark from a projected watermark block of a projectedwatermark block distribution. The projected watermark block distributionis based on the projected coefficient block distribution.

Several advantages may be obtained in embodiments. Embodiments of thepresent invention may duplicate watermark data over an image and alsospread each instance of a watermark over a defined area in order toachieve increased robustness to compression and cropping. Out-of-orderrecovery of watermark data elements may be done in embodiments. Further,strengthening robustness of watermarking against cropping andcompression may be obtained. By separating detection from extraction ofthe watermark, embodiments of the present invention may remarkablyincrease the computational efficiency of the process and also enhancerobustness. For example, embodiments of the present invention can allowthe detection of a watermark to a very high degree of compression and/orcropping, and the extraction of an embedded watermark also to a highdegree of compression and/or cropping. Embodiments of the presentinvention may also embed watermarks that are substantially invisible tothe human eye and therefore do not perceivably distort the originalimage.

In embodiments, watermarking is provided that may be practical,computationally efficient for use in environments with a large number ofimages to be processed, and robust to compression and cropping to be aeffective tool in reducing the misuse of digital images in environmentsincluding the WWW.

Embedding a Watermark

FIG. 1 illustrates a process 100 to embed a watermark in a digitalimage, according to an embodiment of the present invention (steps102-116). In step 102, the image is transformed into a representativenumerical coefficient matrix, for example, a matrix of coefficients inthe discrete cosine transform (DCT) domain. The matrix of coefficientsis referred to hereafter interchangeably as “coefficient matrix” or “DCTmatrix.” The description herein is based on coefficients in the DCTdomain. However, other transformations in the frequency domain thatyield coefficient matrices corresponding to a digital image arepossible, and are contemplated within embodiments of the presentinvention.

FIG. 2 illustrates a coefficient matrix 200 according to an embodimentof the present invention. Coefficient matrix 200 comprises, for example,M×N coefficients, such as coefficient 202. For example, when an image istransformed using the DCT transform, then the coefficients are DCTcoefficients. The M×N coefficients are logically partitioned tocoefficient blocks (CB), such as, coefficient block 204 of apredetermined size, for example, 8×8 coefficients. The coefficientblocks of coefficient matrix 200 are logically grouped intogroup-of-blocks (GOB), such as, GOB 206. For example, four adjacent DCTblocks can be included in a corresponding GOB 206. The GOBs ofcoefficient matrix 200 are then grouped into watermark blocks (WB), suchas, watermark block 208. The number of GOB in each watermark block 208can depend on the characteristics of the watermark being embedded. Thefunction and use of coefficients, coefficient blocks, group-of-blocks,and watermark blocks are explained below with respect to variousprocesses.

Coefficients (e.g., 202) in a coefficient matrix (e.g., 200) cancorrespond to pixels in a digital image. For example, the DCTtransformation assigns a value in the frequency domain for each group ofone or more pixels in a digital image. One method of computing the DCTmatrix of an image, the two-dimensional DCT, is described in Andrew B.Watson, “Image Compression Using the Discrete Cosine Transform,”Mathematica Journal, v 4(1), p. 81-88 (1994), which is herebyincorporated by reference in its entirety.

In step 104, coefficient blocks (e.g., 204) of coefficient matrix (e.g.,200) are identified and a watermark-indicator is embedded in eachcoefficient block. For example, if a coefficient matrix is comprised ofM×N coefficients, then logically partitioning it into 8×8 coefficientblocks will result in M/8×N/8 coefficient blocks. A watermark-indicatorcan be embedded in each of the M/8×N/8 coefficient blocks.

A watermark-indicator can be embedded by replacing a predetermined setof coefficients in each coefficient block, referred to as a“watermark-indicator coefficient set”, with another value. In oneembodiment, in each coefficient block, coefficients of a predetermineddiagonal of coefficients can be set to a new value to indicate thepresence of a watermark in the image. For example, in GOB 206 as shownin FIG. 3, the diagonals 302A, 302B, 302C and 302D of respectivecoefficient blocks 308A, 308B, 308C, and 308D, can be chosen to embedthe watermark-indicator. The coefficients in diagonal 302A, i.e., thewatermark-indicator coefficient set corresponding to coefficient block308A, can be set to the average value of the coefficients of thatdiagonal. Similarly, the coefficients of each of the diagonalscorresponding to 302A in all other coefficient blocks in the coefficientmatrix can be set to the average value of coefficients of the respectivediagonal.

In the example above, the diagonal for embedding the watermark-indicatorin each coefficient block is chosen because the coefficients in thosediagonals, in the DCT transformation of an image, are representative offrequency components that are above the normal perception range and alsosubstantially unaffected by JPEG compression. Other choices ofcoefficients for the watermark-indicator coefficient set in a similarlyrepetitive, easily determinable manner are possible and are contemplatedwithin embodiments of the present invention. It should also be notedthat the coefficients in the watermark-indicator coefficient set can beset to a value other than the average value.

The embedding of a watermark-indicator in a repetitive, easilydeterminable manner contributes to reduced execution times androbustness to cropping and compression that are characteristic ofembodiments of the present invention.

In step 106, the GOB boundary is determined. In one embodiment, a GOBmay contain 4 coefficient blocks arranged in a 2×2 coefficient blockarray, i.e., 2 rows and 2 columns of coefficient blocks in thecoefficient matrix, for example, as shown in FIGS. 2 and 3. Otherchoices for the number and organization of the selected coefficientblocks that comprise a GOB are possible and are contemplated within theteachings in this disclosure.

In step 108, one coefficient block in the GOB is selected to be an“information block”, and another two blocks within the GOB are selectedto be “contributor blocks.” The information block is a coefficient blockthat holds all or part of a watermark. The contributor blocks are othercoefficient blocks in the same GOB as the corresponding informationblock that provides coefficient values based on which the watermark datais embedded in the information block. Coefficients in contributor blocksthat are used for deriving the coefficient values for the watermarkcoefficient set can be referred to as the watermark contributorcoefficient set. In one embodiment, in a GOB 206 of 2×2 coefficientblocks, coefficient block 308D is selected to be the information block,and coefficient blocks 308B and 308C are selected as contributor blocks.Contributor block 308B is referred to as the “top block”, andcontributor block 308C is referred to as “left block”. The selectedpattern is common for all GOBs of the coefficient matrix.

A predetermined number of coefficients are selected in the informationblock (referred to as a “watermark coefficient set”) where their valueswill be modified to encode part of the chosen watermark. For example, inFIG. 3, the coefficient diagonal 304D can be selected to embed a part ofthe watermark. Other choices for the information block and contributorblocks, as well as for the number and organization of the watermarkcoefficient set are possible and are contemplated within the embodimentsof the present invention.

In step 110, the number of GOB required for a WB is determined based onthe watermark and the number of bits of the watermark to be embedded ineach GOB. The predetermined watermark can be divided into one or more“watermark-units”, and each watermark-unit can be embedded in adifferent GOB. For example, if the watermark is a 32-bit string and itis decided to embed 8 bits of the watermark in each GOB, 4 GOBs canconstitute the corresponding WB. Each WB includes at least a sufficientnumber of GOBs to embed a complete instance of the selected watermark.

In step 112, the watermark can be defined to be robust even whenin-order extraction is not guaranteed. For example, when a watermarkedimage has been cropped, it is likely that parts of the watermark arereturned out of order in extracting the watermark from the watermarkedimage. Additional data, referred to as “sentinel-units”, can be insertedat selected points between watermark-units of the predeterminedwatermark.

For example, consider a relatively large image that is likely to becropped and in which a 32-bit watermark is to be embedded. If 4 bits areto be embedded in each GOB, a total of 8 GOBs are required to embed oneinstance of the entire 32-bit watermark. Therefore, the corresponding WBshould include at least 8 GOB. In general, the 4-bit parts (i.e., thewatermark-units) of the 32-bit watermark are embedded in order in thecorresponding GOB. But, if the watermarked image is cropped, the orderin which those 4-bit parts of the watermark are retrieved may not be thesame as the order in which they were embedded. Therefore, sentinel-unitsmay be interspersed at predetermined locations in the ordered sequenceof watermark-units of the watermark. For example, 4 sentinel-units s1,s2, s3, s4 can be interspersed with the 8 watermark-unit sequence of{d1, d2, d3, d4, d5, d6, d7, d8} to have a total of 12 combinedwatermark-units and sentinel-units that constitute the processedwatermark as follows: {s1, d1, d2, d3, s2, d4, d5, d6, s3, d7, d8, s4}.This processed sequence of watermark units, when embedded in a WB of 3×4GOB array, can completely determine the order of the watermark unitseven when the extraction of the watermark units does not preserveordering.

In step 114, the watermark, with or without being processed to preserveordering in extraction, is embedded in the coefficient matrix. In oneembodiment of the present invention, step 114, as illustrated in moredetail in an example process in FIG. 4, can be used to embed thewatermark in the coefficient matrix.

The value of a predetermined number of coefficients in diagonal 304D canbe modified to embed each bit of the selected watermark. For example,coefficient 306D in coefficient diagonal 304D can be set to the averagevalue of corresponding coefficients 306B in the top block and 306C inthe left block, adjusted up or down by a predetermined constant value.If the corresponding bit of the watermark to be embedded is a 1, thenthe average value of the corresponding coefficients from the top blockand the left block is incremented by a predetermined constant Q. If thebit to be embedded is a 0, then the average value of the correspondingcoefficients from the top block and the left block is decremented by apredetermined constant Q.

Coefficients in diagonal 304D that are to have modified values (i.e.,watermark coefficient set) to represent the embedded watermark arepredetermined based on a random selection or based on some othercriteria. In a DCT transform of a digital image, the coefficients indiagonal 304D represent frequency components that can generally bealtered without substantially affecting the perceived image. Followingthe example above, any 4 coefficients in diagonal 304D can be used toembed a corresponding watermark-unit.

Similarly to embedding one watermark-unit in diagonal 304D of GOB 206,the process is repeated to embed the remaining watermark-units of thatwatermark instance in the other GOB of the WB such that an entirewatermark instance is embedded in order in each WB of the coefficientmatrix.

Returning to process 114 as shown in FIG. 4, a watermark can be embeddedin a coefficient matrix as follows. In step 402, a WB is selected. Ingeneral, each WB can be processed in order of its placement within thecorresponding coefficient matrix, for example, starting from the leftedge of the coefficient matrix and moving across horizontally beforemoving to the next row of WB. In step 404, for each WB selected, a GOBwithin that WB is selected. The GOB in each instance can also beselected, for example, in order, starting from the left edge of thecorresponding WB and moving across horizontally before moving to thenext row of GOB of the WB.

In step 406, within the selected GOB, a coefficient is selected to beupdated with a new value, modified according to the watermark data to beembedded. As described previously, the coefficients to be updated, i.e.,the watermark coefficient set, can be predetermined.

In step 408, for one coefficient in the watermark coefficient set, thevalue is set to the average of the coefficients in the correspondingpositions in the top block and the left block. Then, in step 410, forthat coefficient in the watermark coefficient set, depending on whetherthe bit to be embedded is a 0 or a 1, the value is either decremented orincremented, respectively, by a predetermined constant Q.

In step 412, a determination is made as to whether more data for thewatermark-unit being processed needs to be embedded. If yes, processingreturns to step 406 to select the next coefficient to be modified in thecurrent GOB. If no, and if, in step 414, it is found that more GOBs areto be processed in the current WB, processing returns to step 404 toselect the next GOB and the next watermark-unit to be embedded. If nomore GOBs are to be processed in the current WB and it is determined, instep 416, that there are more WBs to be processed, the processing movesto step 402 and selects the next WB to embed an instance of thewatermark. If, in step 416, it is found that there are no more WB to beprocessed, then all WB available in the digital image have beenprocessed to embed an instance of the watermark. At the completion ofprocess 114, all watermark-units of a watermark instance are embedded ineach WB of the coefficient matrix.

FIG. 5 illustrates a process 500 that can be used to determine if animage is watermarked, according to an embodiment of the presentinvention. If the image is found to be already watermarked, process 500enables the watermark to be retrieved. If the image is found to not havea watermark, a watermark is embedded in the image.

In one example, process 500 may be part of a process that processes anentire set of images collected, for example, by a web crawler or otherdevice and deposited in one or more storage areas accessible to process500. In step 501, a digital image, for example, a digital image in JPEGformat is retrieved, for example, from the WWW or other repository ofimages that is accessible to process 500.

In step 502, the digital image is processed to determine if it alreadyhas a watermark embedded. An exemplary method of determining if awatermark is already embedded is described below with respect to FIG. 6.In step 504, process 500 selects one of two branches of processing basedon whether a watermark was detected in step 502. If a watermark isdetected, then processing proceeds to step 510 to extract the watermark.An exemplary method to extract a watermark is described below withrespect to FIG. 6. Otherwise, if no watermark is detected in step 502,processing proceeds to step 506 to embed a watermark. An exemplaryprocess 100 that can be used to embed a watermark was described abovewith respect to FIG. 1.

Detecting a Watermark

FIG. 6 illustrates a process 600 to determine if an image includes adigital watermark, according to an embodiment of the present invention.In step 601, the image is transformed into a coefficient matrix asdescribed previously. Step 601 is necessary only if the coefficientmatrix has not been generated previously by another process or is notavailable to process 600. In step 602, a starting point is determined ina coefficient matrix representative of the digital image beingprocessed. For example, process 600 may consider the coefficient atcoefficient matrix location (0, 0) as the starting point: the firstelement in the tuple (0, 0) representing the row position of thecoefficient in the coefficient matrix, and the second elementrepresenting the column position. Process 600 can commence processing byconsidering coefficient (0, 0) to be the upper left corner of a firstcoefficient block, for example, 8×8 coefficient block. Thus, in step602, coefficient block boundaries are determined. The collection of allcoefficient blocks based on such boundary determination can be referredto as a coefficient block distribution. The initial determination of acoefficient block distribution may also be considered a projectedcoefficient block distribution.

In step 604, based on the assumed coefficient block boundaries, i.e., aprojected coefficient block distribution, the watermark-indicatorcoefficient set in each coefficient block is processed. For example, itmay be predetermined that the diagonal of 6 elements corresponding todiagonal 302A shown in FIG. 3 would constitute the correspondingwatermark-indicator coefficient set.

In each coefficient block, the standard deviation of thewatermark-indicator coefficients set is calculated. For example, thestandard deviation for all six elements of diagonal 302A is calculatedin this step.

In step 606, the percentage of coefficient blocks in which therespective watermark-indicator coefficient sets have a standarddeviation below a predetermined watermark-indicator threshold isdetermined. If the percentage thus determined is greater than thewatermark-indicator threshold, in step 608, the image is determined tocontain a watermark. Otherwise, further processing is required before adetermination can be made that the image contains no watermark.

If the percentage of coefficient blocks determined in step 606 is lessthan the predetermined watermark-indicator threshold, then process 600returns to step 602 and selects a new coefficient block boundary, i.e.,a new projected coefficient block distribution.

Each time process 600 returns to step 602, the coefficient blockboundary, specifically the top-left coefficient of the first coefficientblock in the coefficient matrix, is moved in a manner to cover eachcoefficient in one coefficient block. For example, in this example,process 600 would return to step 602, each time moving the top-leftcoefficient of the coefficient block to the next element in a 8×8 arrayof coefficients anchored at the top left corner of the coefficientmatrix. Thus, each projected coefficient block distribution is based ona new coefficient offset within a 8×8 array of coefficients.

After a new coefficient block boundary is selected, the processingproceeds through steps 604-606. If, in one iteration, it is determinedin step 606 that the percentage of coefficient blocks having a standarddeviation of the watermark-indicator coefficient set that is less than apredetermined watermark-indicator threshold is greater than the desiredpercentage, then in step 608, it is determined that the correspondingimage is watermarked.

On the other hand, if process 600 iterates through an entire coefficientblock, for example, an entire 8×8 array of coefficients, withoutfinding, in step 606, a percentage of coefficient blocks in which thestandard deviation of the watermark-indicator coefficient set is belowthe predetermined watermark-indicator threshold, then it is concludedthat the image is not watermarked. If it is determined that the image isnot watermarked, for example, process 100 may be invoked to embed awatermark in the image.

If it was determined, in step 606, that a percentage of DCT blocks inwhich the standard deviation of the watermark-indicator coefficients isabove a predetermined watermark-indicator threshold, then it isconcluded that the image is watermarked, and a process such as, forexample, process 700 can be used to extract the watermark.

Various embodiments of the present invention are capable of determiningwhether a particular image contains a watermark in a relativelyefficient manner. In a worst case scenario, the detection processiterates over each coefficient of an array the size of one coefficientblock. Also, because the watermark is embedded in every coefficientblock, the detection is remarkably robust even when the watermarkedimage is compressed or cropped.

Extracting a Watermark

FIG. 7 illustrates process 700 that can be used to extract a watermarkfrom an image. The process of watermark extraction is based ondetermining a boundary for WB, i.e., a “watermark block distribution”,and then determining the corresponding “watermark-unit vectors”, i.e.,vector of watermark-units of a watermark instance.

The size of the watermark, i.e., the length of the watermark, can bepredetermined for each application. For example, where embodiments ofthis invention are applied to images posted in an ORKUT socialnetworking site, the associated watermark embedding application and thewatermark retrieving applications can be configured to use a watermarkof a predetermined length based upon the requirements of the ORKUTapplication. For example, a 32-bit watermark may be sufficient to embeda determinative part of the user identifier associated with ORKUT.Similarly, the coefficient block size, GOB size, WB size, positions ofwatermark-indicator coefficients (watermark-indicator coefficient set),and positions of watermark coefficients (watermark coefficient set) canbe known by process 700 when it begins to process an image to retrievethe embedded watermark.

By already having determined whether the image is watermarked using, forexample, process 600, the coefficient block boundary in the coefficientmatrix corresponding to the digital image being processed can beavailable to process 700. In step 702, a starting point for watermarkdetermination and retrieval is determined. In one embodiment, thestarting point for process 700 is selected as the coefficient blockboundary, at the top left of the coefficient matrix, determined by awatermark detection process. For example, process 700 may consider thecoefficient at coefficient matrix location 804 as the starting point:the first coefficient block in the first WB in the coefficient matrix.The size of the watermark, i.e., length of the watermark, can bepredetermined for each application. Thus, in step 702 a WB boundary isdetermined. This can be considered a projected watermark blockdistribution.

In step 704, based on the currently assumed WB boundary, i.e., thecurrent projected watermark block distribution, data vectorscorresponding to the coefficients embedding watermark instance areretrieved from each WB in the image being processed. Based on thecurrent projected watermark block distribution, respective GOBboundaries are determined. For example, in image 800 shown in FIG. 8,when process 700, in one projected watermark block distribution, assumesthe WB boundary based on WB 810, the corresponding GOB boundaries can bedetermined including GOB 808.

From each WB in the projected watermark block distribution, awatermark-unit vector is generated, yielding a watermark-unit vector setfor that projected watermark block distribution of the coefficientmatrix. In step 706, it is determined whether process 700 should iterateover more coefficient block offsets, and if the answer is yes, process700 returns to step 702. In one example, each coefficient block offsetwithin a projected GOB can be tried in sequence.

Steps 702 and 704 are repeated for each projected watermark blockdistribution. Each projected watermark block distribution may bedetermined by considering a different coefficient block offset withinthe limits of one GOB.

In each GOB for the WB that is being currently processed of a projectedwatermark block distribution, each coefficient in the watermarkcoefficient set is compared against the average value of thecorresponding coefficients in the contributor blocks, e.g., top blockand the left block. If the coefficient in the watermark coefficient setis higher than the average value of the corresponding coefficients inthe contributor blocks, then that coefficient can be considered torepresent a ‘1’ in the embedded watermark. If the coefficient in thewatermark coefficient set is lower than the average value of thecorresponding coefficients in the top block and the left block, thenthat coefficient can be considered to represent a ‘0’. The entirewatermark-unit vector set for the image, for a projected watermark blockdistribution, can be generated by repeating this coefficient averagecomparison for each coefficient in the watermark coefficient set in eachGOB of each WB according to the projected watermark block distribution.

In other embodiments, in step 706, steps 702-704 may be iterated basedon a criteria other than coefficient block offsets in sequence, forexample, coefficient block offsets may be tried in a random pattern, orother offsets based on row and column index can be used.

When, in step 706, it is determined that a sufficient number ofwatermark-unit vector sets have been generated, for example, when steps702-704 has been iterated over all of the coefficient block offsetswithin a projected GOB, process 700 proceeds to step 708. In step 708,one of the sets of watermark-unit vectors is selected as representingthe embedded data.

In one embodiment, the selection of one of the sets of watermark data asrepresenting the embedded data may be based on a correlation metriccomputed for each set of watermark-unit vectors. A correlation metriccan, for example, be based on the frequency of occurrence of a majorityelement. In one embodiment, for each GOB the most frequently occurringwatermark-unit can be identified across all of the watermark-unitvectors in the corresponding set of watermark-unit vectors. Thecorrelation metric for that set of watermark-unit vectors can be thenumber of occurrences of the most frequently occurring watermark-unitsummed over all GOBs.

The method described above for determining a correlation metric based onthe frequency of occurrence of watermark-units, has been observed to beremarkably resistant to many of the types of corruption observed in JPGimages, such as, for example, corruption of selected localized areas.Other methods of determining a correlation metric are possible and arecontemplated within the teachings of this disclosure. For example, abit-by-bit correlation can be performed between all of thewatermark-unit vectors in each set to arrive at a correlation metricvalue.

In step 710, the set of watermark-unit vectors with the highestcorrelation is determined to represent the embedded data. For example,the set of watermark-unit vectors having the highest correlation metriccan be selected to represent the embedded data. The actual embeddedwatermark can then be determined from the set of vectors because eachvector represents watermark coefficients in one WB. For example, avector that matches most other vectors can be chosen, and its data unitstaken to represent the embedded watermark.

In cases, where the image was subjected to cropping, it is highly likelythat the chosen vector does not have the watermark-units in proper orderto represent the embedded watermark. If the watermark was embedded afterhaving been processed for out-of-order extraction, for example, byinserting sentinel-units between the actual watermark-units (see, forexample, the description of steps 110 and 112 of process 100 above) thenusing the sentinel-units that were inserted, the actual data units maybe re-ordered to obtain the actual watermark. The ability to extractdata units out of order and yet be able to determine the actualwatermark is a key characteristic making the methods and systemsdisclosed herein robust to cropping.

Other Example Embodiments

In another embodiment, the watermark and the watermark-indicator can beembedded only in one or more selected areas of the image. In someapplications, restricting the embedding of new data may improve theability of the method to withstand corruption of the watermark, and toimprove computational efficiency in embedding and extracting the datamark. For example, one or more of a set of predetermined hot-spots canbe identified in each image to be watermarked. The watermark can beinserted only in those hot-spots. In one embodiment, a face detection oredge detection mechanism can be employed to detect faces or other morespecific hot-spots, for example, eye, mouth, nose etc., and thewatermark and watermark-indicator can be inserted in a region of theimage covering the selected host-spots.

FIG. 9 is an embedding system 900 to embed watermarks in images, todetermine whether an image is watermarked, and/or to extract watermarksfrom images, according to an embodiment of the present invention. System900 may be a general purpose or special-purpose computer including aprocessor 901, a memory 903, a storage 905, an input interface 907, anoutput interface 909, a communications device 930 and a watermarkingmodule 910.

Processor 901 can include one or more microprocessors, digital signalprocessors (DSP), field programmable gate arrays (FPGA) or similarprocessing devices. Memory 903 can include a temporary digital memorydevice such as random access memory (RAM), and storage device 905 mayinclude an persistent storage device such as, but not limited to,hard-disk, flash memory, or read only memory (ROM). Input interface 907may include, but is not limited to, one or more of keyboard, mouse,pen-input device, connection to a network, or connection to an externaldevice. Output device 909 may include, but is not limited to, one ormore of a display, connection to a network, or other external device.Communication device 930 may include a communication bus such as, butnot limited to a device compliant to the Peripheral ComponentInterconnect (PCI) standard or the Extended Industry StandardArchitecture (EISA).

Watermark module 910 includes sub-modules implementing the functionalityof embodiments of the present invention. Watermark module 910 includes acoefficient computation module 912, a watermark embedding module 914, awatermark-indicator embedding module 916, a watermark extractor module918, and a watermark detector module 920. The functionality of each ofthese modules can be implemented in hardware, firmware, software, orusing a combination thereof.

The functionality of these sub-modules can be implemented under thecontrol of processor 901. For example, processor 901 can control thereceiving of an image through input interface 907, the processing ofthat image using one or more modules in watermark module 910, and theoutputting of the processed image through output interface 909. Duringthe processing of images using watermark module 910, for example, theoriginal image and the image being processed may reside in memory 903.Storage 905 may be used, for example, during the processing to storetemporary information, or to store post-processing images andinformation. Modules 912-920 may also be implemented in a distributedmanner on two or more interconnected computers.

Coefficient computation module 912 includes the functionality totransform an image, for example, an image in JPG format, into acoefficient representation in the frequency domain, e.g., to generatethe coefficient matrix for an image. As described earlier, thecoefficient matrix can be a DCT matrix corresponding to the DCTtransformation of the image. Coefficient computation module 912 alsoincludes the functionality to convert a coefficient matrix, before orafter being processed according to an embodiment of this invention, backinto an image format such as JPG. For example, coefficient computationmodule 912 can include capabilities to perform the inverse DCTtransform. Coefficient computation module 912 can include all or part ofthe functionality described with respect to steps 102 and 116 of process100. In one embodiment coefficient computation module 912 is configuredto implement steps 102 and 116 of process 100 as described above.

Watermark embedding module 914 includes the functionality to embed apredetermined watermark in the coefficient matrix representation of animage. For example, watermark embedding module 914 can include all orpart of the functionality described with respect to steps 106, 108, 110,112, and 114 of process 100. In one embodiment watermark embeddingmodule 914 is configured to implement steps 106, 108, 110, 112, and 114of process 100 as described above.

Watermark-indicator embedding module 916 includes the functionality toembed a watermark-indicator in an image, according to an embodiment ofthe present invention. For example, watermark-indicator embedding module916 can include functionality described with respect to step 104 ofprocess 100. In one embodiment, watermark-indicator watermark-indicatorembedding module 916 is configured to implement step 104 of process 100as described above.

Watermark extractor module 918 includes the functionality to extract theembedded actual watermark from an image, according to an embodiment ofthe present invention. For example, watermark extractor module caninclude functionality described with respect to process 700. In oneembodiment, watermark extractor module 918 is configured to implementsteps 702-710 of process 700 as described above.

Watermark detector module 920 includes the functionality to detect thepresence of a watermark in an image, according to an embodiment of thepresent invention. For example, watermark detector module 920, canprocess the image to discover the presence of a sufficient number ofwatermark-indicators. Watermark detector module 920 can includefunctionality described with respect to process 600. In one embodiment,watermark extractor module 920 is configured to implement steps 601-608of process 600 as described above.

FIG. 10 shows example images before and after processing according to anembodiment of the present invention. Image 1001 is an image beforeprocessing, and image 1002 is the same image 1001 after having beenprocessed according to an embodiment of the present invention. In image1002, a 32-bit watermark has been embedded in every 16×16 pixel area,and a watermark-indicator has been included in every 8×8 pixel area. Acomparison of the two images shows that image 1002, which has awatermark embedded according to an embodiment of the present invention,has no perceivable difference from image 1001, i.e., the watermark issubstantially invisible to the human eye.

The Summary and Abstract sections may set forth one or more but not allexemplary embodiments of the present invention as contemplated by theinventor(s), and thus, are not intended to limit the present inventionand the appended claims in any way.

The present invention has been described above with the aid offunctional building blocks illustrating the implementation of specifiedfunctions and relationships thereof. The boundaries of these functionalbuilding blocks have been arbitrarily defined herein for the convenienceof the description. Alternate boundaries can be defined so long as thespecified functions and relationships thereof are appropriatelyperformed.

The foregoing description of the specific embodiments will so fullyreveal the general nature of the invention that others can, by applyingknowledge within the skill of the art, readily modify and/or adapt forvarious applications such specific embodiments, without undueexperimentation, without departing from the general concept of thepresent invention. Therefore, such adaptations and modifications areintended to be within the meaning and range of equivalents of thedisclosed embodiments, based on the teaching and guidance presentedherein. It is to be understood that the phraseology or terminologyherein is for the purpose of description and not of limitation, suchthat the terminology or phraseology of the present specification is tobe interpreted by the skilled artisan in light of the teachings andguidance.

The breadth and scope of the present invention should not be limited byany of the above-described exemplary embodiments, but should be definedonly in accordance with the following claims and their equivalents.

What is claimed is:
 1. A method of embedding information in a digitalimage to create a watermarked digital image, comprising: transformingthe digital image to a set of coefficient blocks having coefficients ina frequency domain; embedding a watermark-indicator in each of aplurality of coefficient blocks of said set of coefficient blocks;embedding a watermark in each of a plurality of watermark blocks togenerate a watermarked digital image, each watermark block in theplurality of watermark blocks having a predetermined number ofcoefficient blocks from said set of coefficient blocks and apredetermined number of group-of-blocks, wherein each group-of-blockshas a predetermined number of coefficient blocks from the set ofcoefficient blocks and wherein embedding the watermark includes dividingthe watermark into a plurality of watermark-units, selecting awatermark-unit from the plurality of watermark-units and encoding theselected watermark-unit in a group-of-blocks in each watermark block,the encoding including selecting a watermark coefficient set in agroup-of-blocks from said group-of-blocks in each watermark block andmodifying coefficients in the watermark coefficient set, the modifyingincluding selecting a watermark contributor coefficient set for thewatermark coefficient set, wherein the watermark contributor coefficientset and the watermark coefficient set are located in one group-of-blocksand calculating each coefficient in the watermark coefficient set basedon the watermark contributor coefficient set; and outputting thewatermarked digital image.
 2. The method of claim 1, wherein thewatermark coefficient set is further modified according to thewatermark-unit to be embedded.
 3. The method of claim 1, wherein saideach watermark-unit includes one or more sentinel-units, wherein thesentinel-units define an order of said each watermark-unit within thewatermark.
 4. The method of claim 1, wherein the embedding thewatermark-indicator includes modifying values of each of the pluralityof coefficient blocks of said set of coefficient blocks.
 5. The methodof claim 1, wherein the transforming the digital image to a set ofcoefficient blocks, comprises: computing a discrete cosine transform(DCT) of the digital image; and representing the DCT of the digitalimage as a set of coefficient blocks.
 6. A system for watermarking adigital image, comprising: at least one processor; an image transformmodule coupled to the at least one processor and configured totransform, using the at least one processor, the digital image to aplurality of coefficient blocks of coefficients in a frequency domain; awatermark embedding module coupled to the at least one processor andconfigured to embed, using the at least one processor, a watermark ineach of a plurality of watermark blocks to generate a watermarkeddigital image, each watermark block in the plurality of watermark blockshaving a predetermined number of coefficient blocks from said set ofcoefficient blocks and a predetermined number of group-of-blocks,wherein each group-of-blocks has a predetermined number of coefficientblocks from the set of coefficient blocks and wherein to embed thewatermark, the watermark embedding modules is further configured todivide the watermark into a plurality of watermark-units, select awatermark-unit from the plurality of watermark-units and encode theselected watermark-unit in a group-of-blocks in each watermark block,wherein to encode the selected watermark-unit, the watermark embeddingmodule is further configured to select a watermark coefficient set in agroup-of-blocks from said group-of-blocks in each watermark block andmodify coefficients in the watermark coefficient set, the modifyingincluding the watermark embedding module being further configured toselect a watermark contributor coefficient set for the watermarkcoefficient set, wherein the watermark contributor coefficient set andthe watermark coefficient set are located in one group-of-blocks andcalculating each coefficient in the watermark coefficient set based onthe watermark contributor coefficient set; and a watermark-indicatorembedding module coupled to the at least one processor and configured toembed, using the at least one processor, a watermark-indicator in eachof a plurality of said coefficient blocks.
 7. The system of claim 6,wherein said each watermark-unit includes one or more sentinel-units,wherein the sentinel-units define an order of said each watermark-unitwithin the watermark.
 8. The system of claim 6, wherein the embeddingthe watermark-indicator includes modifying values of thewatermark-indicator coefficients set.
 9. An article of manufacturecomprising a non-transitory computer readable medium encodinginstructions thereon that, in response to execution by a processor,cause the processor to embed a watermark in a digital image usingoperations comprising: transforming the digital image to a set ofcoefficient blocks having coefficients in a frequency domain; embeddinga watermark-indicator in each of a plurality of coefficient blocks ofsaid set of coefficient blocks; embedding a watermark in each of aplurality of watermark blocks to generate a watermarked digital image,each watermark block in the plurality of watermark blocks having apredetermined number of coefficient blocks from said set of coefficientblocks and a predetermined number of group-of-blocks, wherein eachgroup-of-blocks has a predetermined number of coefficient blocks fromthe set of coefficient blocks and wherein embedding the watermarkincludes dividing the watermark into a plurality of watermark-units,selecting a watermark-unit from the plurality of watermark-units andencoding the selected watermark-unit in a group-of-blocks in eachwatermark block, the encoding including selecting a watermarkcoefficient set in a group-of-blocks from said group-of-blocks in eachwatermark block and modifying coefficients in the watermark coefficientset, the modifying including selecting a watermark contributorcoefficient set for the watermark coefficient set, wherein the watermarkcontributor coefficient set and the watermark coefficient set arelocated in one group-of-blocks and calculating each coefficient in thewatermark coefficient set based on the watermark contributor coefficientset; and outputting the watermarked digital image.
 10. The article ofmanufacture of claim 9, wherein said each watermark-unit includes one ormore sentinel-units, wherein the sentinel-units define an order of saideach watermark-unit within the watermark.
 11. The article of manufactureof claim 9, wherein the embedding the watermark-indicator includesmodifying values of the watermark-indicator coefficients set.